Section: Partnerships and Cooperations

International Initiatives

GAYA: Semantic and Geometric Models for Video Interpretation

We have formed an associate team GAYA, with the primary goal of interpreting videos in terms of recognizing actions, understanding the human-human and human-object interactions. Despite several years of research, it is yet unclear what is an efficient and robust video representation to attack this challenge. In order to address this, GAYA will focus on building semantic models, wherein we learn the video feature representation with limited supervision, and also geometric models, where we study the geometric properties of object shapes to better recognize them. The team consists of researchers from two Inria project-teams (Thoth and WILLOW) and a US university (Carnegie Mellon University [CMU]). It will allow the three teams to effectively combine their respective strengths in areas such as inference and machine learning approaches for vision tasks, feature representation, large-scale learning, geometric reasoning. The main expected outcomes of this collaboration are: effective learnt representations of video content, new machine learning algorithms for handling minimally annotated data, large-scale public datasets for benchmarking, theoretical analysis of objects shapes and contours. Cordelia Schmid and Karteek Alahari are involved in this associate team.

• University of Edinburgh: C. Schmid collaborates with V. Ferrari, associate professor at university of Edinburgh. Vicky Kalogeiton started a co-supervised PhD in September 2013; she is bi-localized between Uni. Edinburgh and Inria. Her subject is the automatic learning of object representations in videos. The collaboration resulted in a joint publication in IEEE PAMI [7]

• MPI Tübingen: C. Schmid collaborates with M. Black, a research director at MPI, starting in 2013. She spent one month at MPI in May 2016. End of 2015 she was award a Humbolt research award funding a long-term research project with colleagues at MPI. In 2016 the project resulted in the development of a large-scale synthetic human action dataset.

• Technion: J. Mairal started a collaboration with Yonina Eldar (Technion) and Andreas Tillmann (Darmstadt university) to develop dictionary learning techniques for phase retrieval. Their collaboration resulted in a paper accepted to the ICASSP'16 conference [22] and a paper accepted to IEEE Transaction on signal processing [12].

• UC Berkeley: This collaboration between Bin Yu, Jack Gallant, Yuval Benjamini, Adam Bloniarz, Yuansi Chen (UC Berkeley), and Julien Mairal (Inria Thoth) aims to discover the functionalities of areas of the visual cortex. We have introduced an image representation for area V4, adapting tools from computer vision to neuroscience data. The collaboration started when Julien Mairal was a post-doctoral researcher at UC Berkeley and is still ongoing.

• Indo-French project EVEREST with IIIT Hyderabad, India, funded by CEFIPRA (Centre Franco-Indien pour la Promotion de la Recherche Avancee). The aim of this project between Cordelia Schmid, Karteek Alahari and C. V. Jawahar (IIIT Hyderabad) is to enable the use of rich, complex models that are required to address the challenges of high-level computer vision. The work plan for the project will follow three directions. First, we will develop a learning framework that can handle weak annotations. Second, we will build formulations to solve the non-convex optimization problem resulting from the learning framework. Third, we will develop efficient and accurate energy minimization algorithms, in order to make the optimization computationally feasible.

• France-Berkeley fund: Julien Mairal was awarded in 2014 a grant from the France-Berkeley fund for a project with Pr. Bin Yu (statistics department, UC Berkeley) on “Invariant image representations and high dimensional sparse estimation for neurosciences”. The award amounts to 10,000 USD, from November 2014 to April 2016. The funds are meant to support scientific and scholarly exchanges and collaboration between the two teams.